This presentation

Quarto: an open-source scientific and technical publishing system

  • What is literate programming
  • Learn about the different possibilities:
  • Render to the output format you want
  • Outsource rendering with Continuous Integration/Development (CI/CD)

Literate programming is the practice of mixing code and descriptive writing in order to execute and explain a data analysis simultaneously in the same document.

— William Landau Developer of {targets}

For a bioinformatician, pure programming is barely happening, we need reports/websites where content and formatting are disconnected

Source: {targets} user manual

Rmarkdown

Predecessor of Quarto.

Version 1 in 2016, the idea was to have text and code in one document, execute code and let the final markdown converted to different formats by pandoc.

Jupyter notebook were also exploring the same concept. Rendering order led sometimes to some frustration

Credit: Artwork by Allison Horst

Markdown

  • Lightweight markup language with a simple syntax

HTML

<!DOCTYPE html>
<html>
<body>

<h1>This is a heading</h1>

<p>This is some text in a <b>paragraph</b></p>

<h2>This is a second level heading</h2>

<ul>
<li><a href="http://exa.com"><code>site</code></a>
<li><img src="https://images.computerhistory.org/revonline/images/500004391-03-01.jpg?w=200">
</ul>
</body>
</html>

Markdown equivalent

# This is a heading

This is some text in a **paragraph**

## This is a second level heading

- [`site`](http://exa.com)
- ![](https://images.computerhistory.org/revonline/images/500004391-03-01.jpg?w=200)

Linus and Tux

See author markdown guide

Quarto

Extended Rmarkdown. Languages and Integrated Development Environment agnostic

Positron

Why use Quarto ?

  • Write detailed analyses that render to beautiful reports
  • Ensure reproducibility
  • Mix programming languages (passing along variables)
  • Excellent documentation
  • Export a single (.qmd) document to various formats (PDF, HTML…)
  • Many extensions
  • Text file that can (should) be tracked by a version control system (like git)

Quarto pipeline

Computation

  • In a fresh and clean session
  • Chunks are evaluated
  • Sequentially from top to bottom

Example for knitr

  • Extracts chunks
  • Interpret/Run them
  • Formats results as markdown
  • Re-inject in the document (MD)

Pandoc

  • Converts markdown to the desired document
    • PDF
    • HTML
    • Word
    • EPUB
    • …

Quarto document: Structure

YAML header

  • Document common variables:
    • title, date, author, …
  • Define format and its specific options
  • Use for autocompletion

Free text in Markdown

  • Markdown syntax to write your descriptions, remarks
  • Literate programming

Code chunks

Code to be interpreted

  • knitr for
  • jupyter for
  • Outputs (tables/plots) are inserted below each

Output examples 1/2

HTML

Word

Output examples 2/2

Typst

PDF (LaTex)

Equations using \(\LaTeX\) syntax

How to add equations

  • Enclose in $ for in-line equations

  • $a^2+b^2=c^2$ renders as \(a^2+b^2=c^2\).

  • Double ($$) for separate equations.

$$G_{\mu v}=8 \pi G (T_{\mu v} + \rho _\Lambda \ g_{\mu v}) $$

renders as:

\[G_{\mu v}=8 \pi G (T_{\mu v} + \rho _\Lambda \ g_{\mu v})\]

Reports

  • Automatic Table of content
  • Author metadata (affiliation, , )
  • Here HTML output, but PDF is only one variable modification away

Books

  • Bundle large chapters into a website
    • Chapters on the left
    • ToC on the right
  • Quarto docs are one
  • Exports to PDF / EPUB etc…

Websites

  • Extensive use for teaching
    • Anthoula
    • R. Krause
  • Building and deployment via Continuous Integration (CI)
    • Gitlab
    • GitHub
  • Automatic listing pages, search engine, navigation bar and smart caching
  • Demo using the BASV53

Dashboards, what we do

Demo simple Quarto doc in VScode

Minimal header

---
format: html
---

---
format: pdf
title: Bioinfo DLSM team 2024
author: Aurélien Ginolhac
date: today
execute: 
  echo: false
---

Quarto Manuscripts

Command-line

❯ quarto create project manuscript dlsm-bioinfo
? Title (dlsm-bioinfo) › Services of the bioinfo DLSM team
Creating project at DLSM/Seminars/2024-quarto-intro/dlsm-bioinfo:
  - Created _quarto.yml
  - Created .gitignore
  - Created index.qmd
  - Created references.bib

Outcome

❯ tree dlsm-bioinfo/
dlsm-bioinfo/
├── dlsm-bioinfo.Rproj
├── index.qmd
├── _quarto.yml
└── references.bib

1 directory, 4 files

PLoS template

Command to add an extensions among many available (+165 references)

quarto add quarto-journals/plos

Quarto Manuscripts, what services we offer

dlsm-bioinfo/
├── bioinfo.qmd
├── dag-atac-seq.png
├── dag-rna-seq.png
├── insert-citation.png
├── orbi-bib-export.png
├── _extensions
│   └── quarto-journals
│       └── plos/
├── gaigneaux.bib
├── ginolhac.bib
├── references.bib
├── plos2015.bst <- bib style
├── _manuscript/ <- outputs
└── _quarto.yml  <- main config

Demo HTML output

PLoS PDF example:

Quarto extensions

  • We saw already some examples
    • PLoS format
    • This unilu-theme
    • Pointer (press )

Compiled here

Quarto Drop: a console in your browser

  • Add in header:
---
revealjs-plugins:
  - drop
---
  • Drop a console of webR or pyodide with or button.
  • Pre-load packages
format:
  revealjs:
    drop:
      engine: webr
      webr:
        packages:
          - ggplot2
          - dplyr

Example:

  • {ggplot2} gets automatically downloaded

Outsource rendering

  • Quarto output files usually rendered on your local machine.
  • For HTML files/website, those are static files.
  • Rendering could be done on someone else computer aka the cloud.
  • CI/CD (Continuous Integration/Continuous Deployment) can help
  • But you must use git , as you do anyway right?
  • Two examples:
    • GitHub Actions
    • Gitlab CI

CI description with Docker/git/github

Some icons from flaticon.com, credit to Valueinvestor

GitHub Actions

.github/workflows/publish.yml
on:
  workflow_dispatch:
  push:
    branches: main
name: Quarto Publish
permissions: 
    contents: write
    pages: write
jobs:
  build-deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Check out repository
        uses: actions/checkout@v4 
      - name: Set up Quarto
        uses: quarto-dev/quarto-actions/setup@v2
        with:
          version: pre-release
      - uses: r-lib/actions/setup-r@v2
      - uses: r-lib/actions/setup-r-dependencies@v2

      - name: Publish to GitHub Pages (and render)
        uses: quarto-dev/quarto-actions/publish@v2
        with:
          path: quarto-intro.qmd
          target: gh-pages
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  • 3 steps:

Gitlab CI

  • More verbose, explicit
.gitlab-ci.yml
image: alpine:latest

stages:
  - build_image
  - build_site
  - deploy

build docker image:
  stage: build_image
  image: docker:20.10.12-dind
  before_script:
    - docker info
    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
  script:
    - 'docker pull $CI_REGISTRY_IMAGE:latest || true'
    - docker build --cache-from $CI_REGISTRY_IMAGE:latest --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA --tag $CI_REGISTRY_IMAGE:latest .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest
  only:
    changes:
      - Dockerfile

build_site:
  stage: build_site
  image: $CI_REGISTRY_IMAGE:latest
  cache:
    key: $CI_JOB_NAME
    paths:
      - _freeze/
      - _site/
  artifacts:
    name: "$CI_JOB_NAME"
    expire_in: 2 days
    paths:
      - _site/
  before_script:
    # https://mastodon.social/@eddelbuettel/111693310455494722
    - |
      Rscript -e "read.dcf('DESCRIPTION', 'Imports') |> 
        tools:::.split_dependencies() |> 
        names() |> 
        setdiff(tools:::.get_standard_package_names()$base) |> 
        install.packages()"
  script:
    - quarto render
  when: always

Deploy to BASV53:
  stage: deploy
  only:
    - main
  dependencies:
    - build_site
  before_script:
    - 'which ssh-agent || ( apk add --update openssh )'
    - eval $(ssh-agent -s)
    - 'which rsync || ( apk add --update rsync )'
    - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
    - mkdir -p ~/.ssh && chmod 700 ~/.ssh
    - 'echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
  script:
    - rsync -azh -e "ssh -p 8022" --delete --filter 'protect /preview' $CI_PROJECT_DIR/_site/ $SSH_HOST:/var/www/basv53/

  • Failed pipeline: no deployment as Building site failed.
  • Most recent pipeline, I modified Dockerfile, thus 3 steps.

Dockerfile

Dockerfile
FROM rocker/r2u:22.04

RUN  apt-get update && \
  apt-get install -y \
  curl netbase \
  zip git \
  libxml2-dev libcurl4-openssl-dev libmagick++-dev \
  && rm -rf /var/lib/apt/lists/*

RUN install.r tidyverse rmarkdown biobase knitr V8 gt gtextras  ggimage ggrepel ggupset BiocManager && \
  installBioc.r deseq2 apeglm

# from https://quarto.org/docs/get-started/
ARG QUARTO_VERSION="1.6.33"
RUN curl -LO https://github.com/quarto-dev/quarto-cli/releases/download/v${QUARTO_VERSION}/quarto-${QUARTO_VERSION}-linux-amd64.deb && \
    apt-get update -qq && apt-get -y install \
    ./quarto-${QUARTO_VERSION}-linux-amd64.deb && rm quarto-${QUARTO_VERSION}-linux-amd64.deb \
    && apt autoremove -y && apt clean -y && rm -rf /var/lib/apt/lists/*

r2u is awesome, offering packages as APT.

  • CRAN as Ubuntu binaries, no compilation
  • {tidyverse} install is < 10 sec

by Dirk Eddelbuettel

Positron in beta since June 2024

Before we stop

Summary

  • What is Quarto (qmd)
  • Basic syntax of Markdown
  • Separate content from rendering
  • render to different output formats

Further reading 📚

Acknowledgments 🙏 👏

  • Anthoula Gaigneaux
  • Eric Koncina

Outside

  • Mine Cetinkaya-Rundel
  • Xie Yihuie
  • Hadley Wickham
  • Christophe Dervieux
  • Artwork by Allison Horst
  • Mickaël Canouil
  • Jenny Bryan

Thank you for your attention!